Image and video processing systems are being optimized for size, weight and power (SWAP) constraints. Because the processing systems are smaller and are designed for minimal power consumption, the underlying hardware and algorithms for conducting the image and video processing must be more effective and efficient. Designing such effective and efficient hardware and algorithms to meet SWAP constraints becomes all the more challenging as demands and industry standards for high quality video output (e.g., 1080p high-definition (HD) streaming) continue to increase.
Distortion correction is an essential function of most optical systems. The video stream is subjected to a number of distortions as it passes through the system, including distortions from the shape of the lens (e.g. “fisheye” effects), chromatic aberration (where different wavelengths of light have different focal lengths), the boresighting of the sensor, and the display optics. All of these effects require distortion correction to geometrically skew the video stream so it is displayed “as the human eye would have seen it.”
Several approaches exist for distortion correction in conventional environments; however these have proven unsuitable for applications with extremely demanding power and latency requirements. Demanding power and latency requirements are typical in many image processing systems (e.g., smart phones, tablets or personal digital assistants (PDAs)). The demand for low-power and low-latency is emphasized in man-portable displays for combat, near-to-eye displays for air traffic control (ATC), or head-mounted displays (HMD) for pilots. Any of these systems simultaneously require pristine video quality, long battery life, and extremely low latency to ensure fastest response to life-threatening situations. Low latency is especially critical to avoid disorientation or nausea for users of real-time, head-worn displays. The processing demands in memory bandwidth and computation are very high, often to the exclusion of system-on-a-chip or similar implementations that would otherwise be chosen to meet low-power constraints. The low latency requirements preclude the use of normal frame buffering in conjunction with the underlying algorithms. Furthermore, other conventional approaches are insufficient when the low latency requirements are combined with a requirement for high video quality.